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(54) Data processor and decoder and encoder for a moving pictures database with high quality 



(57) A data processor includes a first encoder, a first 
storage medium, a subtracter, a second encoder, a sec- 
ond storage medium, and an output section. The first 
encoder generates, by using a predetermined first cod- 
ing method, a main code prepared by encoding a mov- 
ing picture signal input as an original signal. The first 
storage medium stores the main code generated by the 
first encoder. The subtracter calculates a coding distor- 
tion generated upon coding by the first encoder. The 
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second encoder encodes, by using a predetermined 
second coding method, the coding distortion calculated 
by the subtracter, thereby generating a compensation 
code. The second storage medium stores the compen- 
sation code generated by the second encoder. The out- 
put section outputs the main and compensation codes 
respectively stored in the first and second storage 
media. 
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Description 



The present invention relates to a data processor 
for constructing a database of encoded moving picture 
signals, and a receiver which receives an output from 
the data processor. 

The use of the original signals of picture signals, 
and particularly, moving picture signals increases the 
data amount. For this reason, before transmitting an 
original signal without any change or reading/writing it 
between storage media, the original signal is generally 
subjected to compression coding (to be simply referred 
to as coding hereinafter) to eliminate statistical and vis- 
ual redundancies. 

Usually, a picture signal thus encoded is transferred 
via a communication line or stored in a storage medium 
at a fixed transfer or read/write rate called an STM (Syn- 
chronous Transfer Mode). 

In recent years, a new transfer method called an 
ATM (Asynchronous Transfer Mode) capable of instan- 
taneously changing the transfer rate is becoming avail- 
able. At present, however, STM service systems are 
always considered for communication application and 
storage application, and coding at the fixed rate is 
assumed to realize the quality of an encoded picture (to 
be simply referred to as a picture quality hereinafter) 
free from any practical problem. 

Fig. 9 shows a conventional picture database. 
While a moving picture signal 1 input to the database is 
controlled with a parameter by a constant rate controller 
2 so as to keep the transfer rate constant, the picture 
signal is encoded by an encoder 3 and stored in a stor- 
age medium 4. 

As is well known, the spacial and time correlations 
between dynamic pictures often change locally, instan- 
taneously, and abruptly. For example, the information 
amount generated by lossy coding generally called 
high-efficiency coding, such as H.261 serving as an 
international standard coding scheme for a video phone 
and a video conference picture signal, and MPEG-1 and 
MPEG-2 serving as international standard coding 
schemes used for the purpose of storage/communica- 
tion/broadcast, greatly changes depending on the 
redundancy of an input picture. 

To stably realize a high picture quality at a fixed 
transfer rate for various input pictures having different 
redundancies, the transfer rate must be fixed high, 
which is not a proper method in practice At present, the 
picture quality is slightly sacrificed to fix the transfer rate 
lower and simplify the system configuration. For eco- 
nomical advantages, the rate to be fixed is set as low as 
possible. 

A typical example of the transfer rate in the use of 
high-efficiency coding is a case wherein an NTSC color 
TV signal is encoded using MPEG-2 while maintaining a 
good picture quality. At this time, a proper transfer rate 
is said to be 6 Mb/s for each channel. At present, the 
digital television picture transfer technology is being 



introduced into satellite broadcasting, terrestrial broad- 
casting, a cable television, and the like, the transfer rate 
is one of the most important parameters in system 
design. 

s When coding at a fixed rate results in an insufficient 
picture quality, if the coding rate can be intermittently 
increased, a higher picture quality can be attained as a 
whole in many cases. Generally speaking, coding is 
usually performed at a f ixed rate to output "main infor- 
10 mation", and when the picture quality is unsatisfactory 
with only the main information, "compensation informa- 
tion" that reduces a distortion caused by coding (coding 
distortion) is to be added, thereby greatly improving the 
picture quality. 

is One example of the prior arts employing this idea is 
"TV SIGNAL CODING EQUIPMENT (Prior Art 1) dis- 
closed in Japanese Patent Laid-Open No. 51-117820. 
According to this technique, as shown in Fig. 10. a tem- 
porarily encoded picture (encoded information e^ is 
20 decoded (into a decoded signal /) on the transmission 
side, a difference (coding distortion information e 2 ) 
between the original picture (input picture signal y) and 
the decoded signal is also encoded and multiplexed 
with the main encoded information e 1( and the obtained 
25 signals are transferred via a single transfer path. The 
coding distortion information e 2 is "compensation infor- 
mation". On the reception side, the encoded information 
et is decoded into the decoded picture signal y\ and 
compensation is performed based on the coding distor- 
30 tion information e 2 to output a picture signal y". The 
degree of picture quality is higher in the order of y, y". 
and y\ 

Another similar prior art is "CODING AND DECOD- 
ING DEVICE FOR ANIMATING PICTURE SIGNAL" 
35 (Prior Art 2) disclosed in Japanese Patent Laid-Open 
No. 61-214885. Compared to Prior Art 1. this technique 
expands to a case using both motion-compensated 
interframe prediction and orthogonal transform. 

In both Prior Arts 1 and 2, when the picture quality 
40 is unsatisfactory, "compensation information" is added 
to "main information" and transferred in order to reduce 
a coding distortion. However, to improve the picture 
quality improved as in Prior Arts 1 and 2 on the assump- 
tion of transfer by the STM as a general transfer form at 
45 present, the transfer rate must be determined on the 
basis of the information amount upon adding "compen- 
sation information", and a higher transfer rate is 
required. 

In both Prior Arts 1 and 2, data generated by coding 
so is formed as data in which "main information" and "com- 
pensation information" are integrally inseparable from 
each other. That is. the information made up of the two 
kinds of information corresponds to substantial "main 
information". For this reason, a database constructed by 
55 only conventional "main information" cannot be used 
without any change, and a database must be recon- 
structed with original picture signals. 

It is an object of the present invention to provide to 
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provide a database processor and a receiver for con- 
structing a database with a higher picture quality in a 
dynamic picture database storing encoded picture infor- 
mation while utilizing picture data stored in a conven- 
tional database. 

In order to achieve the above object, according to 
the present invention, there is provided a data proces- 
sor comprising first coding means for generating, by 
using a predetermined first coding method, a main code 
prepared by encoding a moving picture signal input as 
an original signal, first storage means for storing the 
main code generated by the first coding means, coding 
distortion calculation means for calculating a coding dis- 
tortion generated upon coding by the first coding 
means, second coding means for encoding, by using a 
predetermined second coding method, the coding dis- 
tortion calculated by the coding distortion calculation 
means, thereby generating a compensation code, sec- 
ond storage means for storing the compensation code 
generated by the second coding means, and output 
means for outputting the main and compensation codes 
respectively stored in the first and second storage 
means. 

Brief Description of the Drawings 

Fig. 1 is a block diagram of a data processor 
according to the first embodiment of the present 
invention in which a coding distortion information 
database is added to an existing dynamic picture 
database; 

Fig. 2 is a block diagram of a data processor 
according to the second embodiment of the present 
invention in which two databases for a main signal 
and a coding distortion are simultaneously formed; 
Fig. 3 is a block diagram of a data processor 
according to the third embodiment of the present 
invention; 

Fig. 4 is a block diagram of a data processor 
according to the fourth embodiment of the present 
invention; 

Fig. 5 is a block diagram for explaining the first 
example of supplying encoded data from the data- 
base according to the present invention; 
Fig. 6 is a block diagram for explaining the second 
example of supplying encoded data from the data- 
base according to the present invention; 
Fig. 7 is a block diagram of a receiver according to 
the fifth embodiment of the present invention; 
Fig. 8 is a block diagram of a receiver according to 
the sixth embodiment of the present invention; 
Fig. 9 is a block diagram of a conventional data 
processor storing a picture database; and 
Fig. 10 is a block diagram of a conventional data 
processing system for transferring coding distortion 
information. 



Description of the Preferred Embodiments 

The present invention will be described in detail 
below with reference to the accompanying drawings, 
s Fig. 1 ^hows an arrangement of the first embodi- 

ment when the present invention is applied to a main 
data processor in which a moving picture signal 1 1 has 
already been encoded based on a predetermined cod- 
ing method, and stored as a database in a storage 
w medium 13. In this embodiment, additional information 
is newly added to a main data processor storing a data- 
base which is conventional used, thereby improving the 
picture quality provided to the database user. 

In the main data processor shown in Fig. 1 , an input 
is original signal (moving picture signal 1 1 ) is encoded into 
a main code by an encoder 12 and stored in the storage 
medium 13 in advance. The encoder 12 and the storage 
medium 13 constitute the existing main data processor. 
In this embodiment, an auxiliary data processor having 
20 the following arrangement is connected to the existing 
main data processor. 

A decoder 14 reads the main code from the storage 
medium 13, and decodes it into a main signal. The 
decoded main signal generally contains a coding distor- 
25 tion, and a difference between the decoded main signal 
and the original signal 1 1 is calculated by a subtracter 
17. This difference is a coding distortion, and is not zero 
in general. This coding distortion is encoded by an 
encoder 1 5 using the same coding method as that of the 
30 encoder 12, and stored in a storage medium 16. This 
processing is performed for all main codes stored in the 
storage medium 13. The coding method used in the 
encoder 15 may be part of the coding method used in 
the encoder 12. 
35 The operation rates of the decoder 14 and the 
encoder 15 are not unique. For example, when calcula- 
tion of a coding distortion is executed using a computer, 
the rates of signal supply from both the storage medium 
13 and the original signal 1 1 are easily matched to the 
40 computer. For this purpose, a large-capacity storage 
medium which can be easily accessed from the compu- 
ter may be used for both the storage medium 1 3 and the 
original signal 1 1. As for the input moving picture signal 
1 1 , a digital VTR is usable recently, which often incorpo- 
45 rates a rate adjustment buffer memory in order to 
exchange picture data with the computer. 

The encoder 15 need not always operate in "real 
time". A picture is encoded by the computer slower than 
"real time", and the encoded picture is stored in the stor- 
so age medium 16. With this processing, a high picture 
quality can be realized only by newly adding an auxiliary 
database without changing a conventional database. In 
this case, the "real time" operation means an operation 
using a TV signal standardization frequency of. e.g., 
55 about 1 0 to 1 5 MHz. but a detailed operation rale is not 
limited to this. At the operation rate of the computer, a 
moving picture signal such as a TV signal is generally 
difficult to process in "real time". More specifically, when 
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the TV signal is an NTSC signal, it generates about 30 
pictures per second. Encoding all the 30 pictures within 
one second requires an ability substantially equal to that 
of a supercomputer. 

The dynamic picture database has two output sys- 
tems. In this embodiment, for example, for a special 
user who pays a predetermined additional charge, out- 
put 2. i.e., coding distortion information in addition to 
output 1 is sent to provide a high-quality picture service. 
For a general user who does not pay an additional 
charge, only output 1 is supplied. In this manner, a 
dynamic picture database in a service form can be pro- 
vided. 

The first embodiment has exemplified the case 
wherein the auxiliary data processor according to the 
present invention is applied to the main data processor 
storing an existing picture database. When the latest 
picture not stored in an existing database, such as the 
latest movie, is to be newly encoded, the whole system 
must be operated in real time for moving picture signals 
output from a medium operating in real time such as a 
VTR. In the second embodiment of the present inven- 
tion, the first coding and the second coding are per- 
formed in real time. 

All of encoders 22 and 25, a decoder 24, and a sub- 
tracter 27 shown in Fig. 2 according to the second 
embodiment of the present invention are operable in 
real time. At the same time a moving picture signal 21 is 
input, the moving picture signal 21 and a coding distor- 
tion from the subtracter 27 are encoded, and the results 
are respectively stored in storage media 23 and 26. In 
this case, the storage media 23 and 26 require realtime 
input buffer memories 23a and 26a. 

Depending on applications other than the above 
case, this processing must be executed in real time. For 
example, a live broadcast such as a sports program 
inevitably requires realtime processing. 

Fig. 3 shows the third embodiment of the present 
invention. A moving picture signal supplied from a mov- 
ing picture signal source 31 is controlled with a coding 
parameter by a constant rate controller 35 so as to keep 
the coding rate (transfer rate in the transfer system) 
constant encoded by an encoder 32, and stored in a 
storage medium 33. Assume that this processing has 
been performed in advance at a past point, and the stor- 
age medium 33 functions as a constructed database. 

For the descriptive convenience, an application of 
the present invention to a specific work (work A) will be 
explained. Work A encoded and stored in the storage 
medium 33 is read at a rate (real time) necessary for TV 
signal processing, while being decoded by a decoder 
34. At this time, a subtracter 39 calculates a difference 
between work A and an input signal supplied at an 
adjusted timing to obtain a coding cfistortion. This cod- 
ing distortion is encoded by an encoder 36 undetsuch 
control as to keep its quality constant by a constant 
quality controller 38, and stored in a storage medium 
37. This database system supplies both output 1 and 



output 2, which have the main-compensation informa- 
tion relationship. 

Practically, the storage medium 33 cannot always 
operate in real time. In this case, the system must be 
5 devised such that a difference (coding distortion) from 
an input signal from the moving picture signal source 31 
is calculated when an output from the storage medium 
33 is decoded by one frame. However, this is not essen- 
tial in the present invention. 
10 As for the number of works, not only work A but also 
an enormous number of works are generally stored. As 
needed, the processing for work A is repeatedly exe- 
cuted for all the stored works. 

Coding control for a constant picture quality is eas- 
15 ily realized. Normally, quantization characteristics 
should be fixed for a prediction error. In the present 
invention, quantization characteristics particularly in 
coding of a coding distortion are fixed. In Fig. 10, the 
quantization characteristics mean processing for adjust- 
20 ing the number of possible levels for prediction errors, 
and controlling an information generation situation. In 
general, this processing is the most effective for control 
of the generated information amount that limits the 
number of possible states. 
25 In an actual service, both of these outputs are sent 
to the user, and used to perform decoding, thereby 
obtaining a picture with a higher quality than that in 
decoding only output 1 . 

The fourth embodiment exemplifying a case 
30 wherein the system operates in real time will be 
explained with reference to Fig. 4. A description of the 
same functions as those in Fig. 3 will omitted. At the 
same time a moving picture signal supplied from a mov- 
ing picture signal source 41 in real time is encoded by 
35 an encoder 42, a decoded signal is generated by a 
decoder 44, and a difference (coding distortion) there- 
between is generated in a subtracter 49. When data is 
only supplied without being stored, storage media 43 
and 47 can be skipped to supply outputs 1 and 2. Refer- 
40 ence numeral 46 denotes an encoder; and 48. a con- 
stant quality controller. 

The difference between Figs. 3 and 4 is that a cod- 
ing distortion for either past encoded data stored in a 
database or currently encoded output data is obtained 
45 in generating a coding distortion. In Fig. 3, it is generally 
very difficult to construct a database in the same data- 
base as the conventional one. so that a physically differ- 
ent auxiliary database is used. In Fig. 4. a database is 
easily constructed in a single system. 
so Fig. 5 shows the first example when encoded data 
are supplied from two databases shown in Fgs. 1 to 4. 
An output section 120 supplies an output from a storage 
medium 113 via a communication path 101. An output 
section 130 supplies an output from a storage medium 
55 1 1 6 via a gate 1 03 whose output is ON/OFF-controlled 
by a gate controller 104. and a communication path 
102. The storage medium 1 13 corresponds to the stor- 
age medium 13. 23. 33. or 43, whereas the storage 
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medium 1 16 corresponds to the storage medium 16. 26, 
36, or 46. 

Under the ON/OFF output control of the gate con- 
troller 104 for coding distortion information, output 2 
(coding distortion information) is transmitted to a special 
user who pays an additional charge for, e.g.. a pay serv- 
ice, while output of the coding cfistortion information is 
stopped for a normal user. Alternatively, instead of con- 
trolling the gate 103, execution/nonexecution of setting 
of the communication path 102 may be controlled. 

This control can be executed in not the gate 1 03 but 
a user's receiver. In this case, the service provider 
sends a reception enable signal to a gate function 
installed in the user's receiver. 

The special user means a user who can receive 
information about a coding distortion and decode a 
high-quality picture. The normal user means a user who 
can receive and decode only main encoded data. 

Fig. 6 shows the second example when encoded 
data are supplied from two databases. In multiplexing 
two outputs from storage media 113 and 1 1 6 at a switch 

105 in an output section 140, a multiplexing controller 

106 instructs execution/nonexecution of multiplexing. 
That is, multiplexing of two outputs is executed for the 
special user, and only an output from the storage 
medium 1 13 is supplied to the normal user via a com- 
munication path 107. 

Fig. 7 shows the arrangement of a receiver accord- 
ing to the fifth embodiment of the present invention. 
Main and compensation codes supplied via two com- 
munication paths 101 and 102 are respectively supplied 
to decoders 108 and 109 and decoded. 

Not in a reception state as the special user, the 
decoder 109 is instructed to stop its operation by a con- 
trol signal S. and outputs zero data in place of a coding 
distortion. In an adder 1 10, the coding distortion or zero 
data is added with a main signal from the decoder 108, 
and supplied to a postprocessing unit 111. 

The postprocessing unit 111 basically performs 
scheme conversion into NTSC, PAL/SECAM, or the like, 
and processing for outputting and displaying data on a 
TV monitor. 

If the system has a function of reducing/removing 
noise within a picture caused by coding, the gist of the 
present invention is not influenced. The special user 
can obtain coding distortion information from the 
decoder 109, whereas only zero data is output from, 
e.g., the decoder 109 to the normal user. 

Fig. 8 shows the arrangement of an apparatus on 
the reception side according to the sixth embodiment of 
the present invention. Fig. 8 shows the arrangement of 
a receiver when two outputs from the databases shown 
in Fig. 6 are multiplexed. Two multiplexed outputs sup- 
plied via a communication path 107 are separated by a 
separator 112, and input to decoders 114 and 115. 
respectively. 

Not in a reception state as the special user, the 
decoder 115 is instructed to stop its operation by the 



control signal S, and outputs zero data in place of a cod- 
ing distortion. In an adder 117, the coding distortion or 
zero data is added with a main signal from the decoder 
114, and supplied to a postprocessing unit 118. 
5 As has been described above, in constructing a 
dynamic picture database storing many encoded mov- 
ing picture signals, the currently used method of storing 
encoded pictures at a fixed rate is not preferable 
because the picture quality varies depending on the 
10 spatial, time, and statistical properties of an input pic- 
ture signal. In the future, higher<|uality services will be 
gradually important, so the current method will become 
apparently unsatisfactory. 

According to the conventional database construc- 
ts tion method, coding distortion information correspond- 
ing to "compensation information'' must be added to 
attain a higher-quality picture. Nevertheless, this infor- 
mation cannot be added due to structural limitations. A 
dynamic picture database of encoded pictures at a 
20 higher coding rate must be reconstructed. 

According to the present invention, no "compensa- 
tion information" is added to a moving picture signal 
having a satisfactory picture quality at a conventional 
coding rate, and a database storing -compensation 
25 information" is additionally constructed for only a mov- 
ing picture signal in which a decrease in picture quality 
is recognized. In outputting information from the data- 
base, it suffices to supply main information and the 
"compensation information" parallel or upon multiplex- 
30 ing. An "oteT database for the main information can be 
continuously used. Also, an old receiver which can meet 
with only the old database, a new receiver which 
receives information from the old database, and a new 
receiver which receives information from the new data- 
35 base can be used without any change. 

An old receiver which receives information from the 
new database can be continuously used only by ignor- 
ing "compensation information". 

In this manner, the effects of practical use of the 
40 present invention are very large. In the present inven- 
tion, the method of encoding a "coding distortion" need 
not be the same as the method of encoding a signal. A 
fixed transfer rate need not be particularly used for the 
communication path through which the "coding distor- 
ts tion" is transferred. Even a variable line such as an ATM 
is usable, and the present invention is still effective in 
such an application. 
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Claims 

1 . A data processor characterized by comprising: 

first coding means (12. 22, 32, 42) for generat- 
ing, by using a predetermined first coding 
method, a main code prepared by encoding a 
moving picture signal input as an original sig- 
nal; 

first storage means (13, 23. 33. 43) for storing 
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the main code generated by said first coding 
means; 

coding distortion calculation means (17. 27, 37, 
47) for calculating a coding distortion gener- 
ated upon coding by said f irst coding means; 
second coding means (15. 25. 35. 45) for 
encoding, by using a predetermined second 
coding method, the coding distortion calculated 
by said coding distortion calculation means, 
thereby generating a compensation code; 
second storage means (16. 26, 36. 46) for stor- 
ing the compensation code generated by said 
second coding means; and 
outputmeans (120. 130. 140) for outputting the 
main and compensation codes respectively 
stored in said first and second storage means. 

A processor according to claim 1. further compris- 
ing decoding means (14. 34) for receiving and 
decoding the main code stored in said first storage 
means, and 

wherein said coding distortion calculation 
means calculates the coding distortion on the 
basis of a difference between the main signal 
obtained upon decoding by said decoding 
means, and the original signal. 

A processor according to claim 1. further compris- 
ing decoding means (24. 44) for receiving and 
decoding the main code encoded by said first cod- 
ing means, and 

wherein said coding distortion calculation 
means calculates the coding distortion on the 
basis of a difference between the main signal 
obtained upon decoding by said decoding 
means, and the original signal. 

\ A processor according to any of claims 1 to 3, 
wherein the second coding method is the same as 
the first coding method. 

; A processor according to any of claims 1 to 3, 
wherein the second coding method is part of the 
first coding method. 



7 A processor according to claim 6, wherein said sec- 
ond output means comprises compensation code 
output control means (103. 104) for controlling 
whether the compensation code stored in said sec- 
ond storage means is output, independently of out- 
put of the main code by said first output means. 

8 A processor according to any of claims 1 to 5. 
" wherein said output means comprises multiplexing 

means (105) for multiplexing the main code stored 
in said f irst storage means, and the compensation 
code stored in said second storage means in corre- 
spondence with the main code, and 

said output means outputs the main and com- 
pensation codes multiplexed by said multiplex- 
ing means to a physically common 
communication path (107). 
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6. A processor according to any of claims 1 to 5. 
wherein said output means comprises 

first output means (1 20) for outputting the main 
code stored in said first storage means to a first 
communication path (101). and 
second output means (130) for outputting the 
compensation code stored in said second stor- 
age means to a second communication path 
(1 02) logically independent of the first commu- 
nication path. 



so 9. A processor according to claim 8. further compris- 
ing multiplexing control means (106) for controlling 
said multiplexing means to select whether the mul- 
tiplexed main and compensation codes are output 
or only the main code is output. 

10 A data processor characterized by comprising a 
' main data processor (12. 13) which receives a mov- 
ing picture signal as an original signal, and stores a 
main code generated by performing lossy coding 
for the original signal, and an auxiliary data proces- 
sor connected to said main data processor and 

used, . . 

said auxiliary data processor comprising: 

decoding means (14) tor receiving the main 
code from said main data processor, and 
decoding the main code into a main signal; 
cording distortion calculation means (17) for 
receiving the original signal from said mam 
data processor, and calculating a coding distor- 
tion from a difference between the original sig- 
nal and the main signal; 
coding means (15) for encoding, by using a 
predetermined coding method, the coding dis- 
tortion calculated by said coding distortion cal- 
culation means, thereby generating a 
compensation code; 

storage means (16) for storing the compensa- 
tion code generated by said coding means; and 
output means (1 20, 130, 1 40) for outputting the 
compensation code stored in said storage 
means. 
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1 1. A data receiver characterized by comprising: 

f irst decoding means (108) for decoding a main 
code prepared by encoding a moving picture 
signal serving as a main signal input via a first 
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communication path (101) into a main signal; 
second decoding means (109) for decoding a 
compensation code prepared by encoding a 
coding distortion of the main signal input via a 
second communication path (102) into the cod- s 
ing distortion; and 

main signal compensation means (110) for 
generating a compensated main signal from 
the main signal decoded by said first decoding 
means and the coding distortion decoded by 10 
said second decoding means. 

12. A receiver according to claim 1 1 , wherein said sec- 
ond decoding means selects based on an external 
control signal whether the coding distortion input is 
via the second communication path is decoded. 

13. A data receiver characterized by comprising: 

separation means (112) for separating a main 20 
code prepared by encoding a moving picture 
signal serving as a multiplexed main signal 
input via a communication path (107), and a 
compensation code prepared by encoding a 
coding distortion of the main signal; 25 
first decoding means (114) for decoding the 
main code separated by said separation 
means into the main signal; 
second decoding means (115) for decoding the 
compensation code separated by said separa- 30 
tion means into the coding distortion; and 
main signal compensation means (117) for 
generating a compensated main signal from 
the main signal decoded by said first decoding 
means and the coding distortion decoded by 35 
said second decoding means. 

14. A receiver according to claim 13, wherein said sec- 
ond decoding means selects based on an external 
control signal whether the coding distortion sepa- 40 
rated by said separation means is decoded. 
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